Moving body and system

ABSTRACT

A moving body includes processing circuitry. The processing circuitry is configured to collect an external image of the moving body from an external sensor and collect information associated with an inner state of the moving body from an internal sensor. The processing circuitry is configured to determine whether or not to adopt the external image collected by the external sensor as a first image for estimating a position of the moving body, based on the information associated with the inner state of the moving body collected by the internal sensor. The processing circuitry is configured to estimate a position of the moving body by comparing the first image and a second image associated with a collection position of the first image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromthe Japanese Patent Application No. 2019-207099, filed Nov. 15, 2019,the entire contents of which are incorporated herein by reference.

FIELD

Embodiments described herein relate to a moving body and a systemincluding the same.

BACKGROUND

Development of a moving body for automatic conveyance of a cage truck ata distribution warehouse is in progress. A moving body receives a cagetruck loaded with packages at a designated position, autonomouslycarries a received cage truck to a designated position and displays thecage truck at a designated position. Among these operations, in order toaccurately perform the operations for receiving the cage truck anddisplaying the cage truck, an accurate position estimate of the movingbody becomes necessary.

As a method for estimating the position of the moving body, there existsthe use of an obtained image by a camera attached to a moving body. Forexample, in one method, a position information database which stores animage captured at a target position and an image captured at a pluralityof positions in a traveling area are prepared in advance, and theposition of the moving body is estimated by referring to an image storedin the position information database and an image obtained by the cameraof the moving body.

When estimating the position by referring to an image, if a captureconditions at the time of capture, such as a shaking camera etc. are notconsidered, the image quality of the image to be referred will beunstable and the degree of estimation accuracy for the position willdecrease.

The embodiment provides a moving body and a system including the samewhere the estimation accuracy of the position is improved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing a configuration of one example of a systemaccording to a first embodiment.

FIG. 2 is a diagram showing one example of a hardware configuration of aposition estimation device.

FIG. 3 is a diagram showing an arrangement example of a stereo camera asone example of an external sensor.

FIG. 4 is a flowchart indicating a process when preparing a referencedictionary.

FIG. 5 is a diagram showing an example of capture positions of an imageto be registered to the reference dictionary in accordance with theprocess of FIG. 4 .

FIG. 6 is a flowchart indicating a process during position estimation.

FIG. 7A is a diagram showing an example when feature point matching isperformed at images with different capture conditions.

FIG. 7B is a diagram showing an example when feature point matching isperformed at images with similar capture conditions.

FIG. 8 is a diagram showing a configuration of one example of a systemaccording to a second embodiment.

FIG. 9 is a diagram showing a detection example of a defective part by aprocessing unit.

DETAILED DESCRIPTION

According to one embodiment, a moving body includes processingcircuitry. The processing circuitry is configured to collect an externalimage of the moving body from an external sensor and collect informationassociated with an inner state of the moving body from an internalsensor. The processing circuitry is configured to determine whether ornot to adopt the external image collected by the external sensor as afirst image for estimating a position of the moving body, based on theinformation associated with the inner state of the moving body collectedby the internal sensor. The processing circuitry is configured toestimate a position of the moving body by comparing the first image anda second image associated with a collection position of the first image.

Hereinafter, embodiments will be described with reference to thedrawings.

First Embodiment

FIG. 1 is a diagram showing a configuration of one example of a systemaccording to a first embodiment. The system includes a positionestimation device 1, and a moving body 50. The position estimationdevice 1 is configured to be communicable with the moving body 50.

The moving body 50 includes a moving body control unit 51, and a sensorinformation collection unit 52. Further, though not shown in thedrawings, the moving body 50 includes a drive mechanism necessary formoving.

The moving body control unit 51 performs control associated with themovement etc. of the moving body 50. The moving body control unit 51,for example, performs control of the drive mechanism for moving themoving body 50 to a designated target position.

The sensor information collection unit 52 includes various types of aninternal sensor and an external sensor installed in the moving body 50,and collects various sensor information. The internal sensor is a sensorfor enabling the moving body 50 to collect sensor information associatedwith its own state. In the embodiment, the internal sensor mainlycollects sensor information associated with movement of the moving body50. On the other hand, the external sensor is a sensor which collectssensor information associated with a state outside the moving body 50.In the embodiment, the external sensor mainly collects an image outsidethe moving body 50.

The position estimation device 1 is configured to estimate a position ofthe moving body 50 from an image collected by the moving body 50. Theposition estimation device 1 may take the form of a superior system forcontrolling movement of the moving body 50 based on an estimation resultof the position of the moving body 50. The position estimation device 1may be for example, a personal computer (PC). The position estimationdevice 1 includes a sensor information processing unit 10, a sensorinformation acquiring unit 21, a position estimation unit 22, a controlunit 23, and a dictionary storage unit 30.

The sensor information acquiring unit 21 acquires sensor informationfrom the sensor information collection unit 52 of the moving body 50.For example, the sensor information acquiring unit 21 extracts sensorinformation from a signal communicated from the moving body 50.

The sensor information processing unit 10 processes sensor informationacquired at the sensor information acquiring unit 21. The sensorinformation processing unit 10 includes an evaluation informationcalculating unit 11, and a determination unit 12.

The evaluation information calculating unit 11 calculates evaluationinformation for determining by the determination unit 12, based onsensor information of the internal sensor of the moving body 50 acquiredby the sensor information acquiring unit 21. The evaluation informationis, e.g., information associated with a movement of the moving body 50,and includes for example, velocity, angular velocity, and accelerationof the moving body 50.

The determination unit 12 determines whether or not an image outside themoving body 50 acquired by the sensor information acquiring unit 21 canbe adopted for the estimation of the position of the moving body 50. Thedetermination unit 12 determines to adopt, for the estimation of theposition of the moving body 50, an image of evaluation information withsimilar evaluation information at the time of collecting images storedin the dictionary storage unit 30.

The position estimation unit 22 estimates a position of the moving body50 based on the image determined to be adopted for the estimation of theposition by the determination unit 12 out of the images outside themoving body 50 acquired by the sensor information acquiring unit 21, andthe images registered to the dictionary storage unit 30.

The control unit 23 controls the operation of the moving body 50 inaccordance with the position of the moving body 50 estimated at theposition estimation unit 22.

The dictionary storage unit 30 stores a dictionary for storing an imageof a target position of the moving body 50, images collected at aplurality of capture positions near the target position, feature pointsand feature amounts extracted from each image, the degree ofcorrespondence between each image, information on the collectionposition for each image, and evaluation information for each image.

FIG. 2 is a diagram showing one example of a hardware configuration of aposition estimation device. The position estimation device 1, forexample, includes a CPU 101, an input device 102, a display 103, acommunication device 104, a storage 105 as hardware. The CPU 101, theinput device 102, the display 103, the communication device 104, and thestorage 105 are connected to a bus 106.

The CPU 101 is a processor for controlling the entire operation of theposition estimation device 1. The CPU 101, by executing a program storedin, e.g., the storage 105, operates as the sensor information processingunit 10, the sensor information acquiring unit 21, the positionestimation unit 22, and the control unit 23. The CPU 101 may be replacedwith MPU, GPU, ASIC, FPGA etc. The CPU 101 may be a single CPU etc. or aplurality of CPU etc.

The input device 102 is an input device such as a joystick, a touchpanel, a keyboard, and a mouse etc. When the input device 102 isoperated, a signal corresponding to the operation matter is input to theCPU 101 via the bus 106. The CPU 101 performs various processes inaccordance with this signal.

The display 103 is a display such as liquid crystal display or organicEL display. The display device 103 displays various images.

The communication device 104 is a communication device for performing,for example, a wireless LAN communication. The communication device 104communicates with the moving body 50. The communication device 104 neednot necessarily be a communication device for wireless LANcommunication.

The storage 105 is a storage device such as a hard disk drive or a solidstate drive. The storage 105 operates as the dictionary storage unit 30.Further, the storage 105 stores various programs executed by the CPU101.

The bus 106 is a data transfer path for data exchange between the CPU101, the input device 102, the display 103, the communication device104, and the storage 105.

The following explains in more detail the matters regarding eachconfiguration shown in FIG. 1 . The moving body 50 shown in FIG. 1 isfor example, an automated guided vehicle (AGV). The AGV is a tracklessvehicle which automatically travels to a designated package loadinglocation, and carries the package loaded by the person or automatedrobots etc. at the package loading location to a designated unloadinglocation. The moving body 50 includes various drive mechanisms. Forexample, the moving body 50 may include a two-wheeled drive mechanism, afour-wheeled drive mechanism, or a crawler (caterpillar) mechanism.Further, the moving body 50 may be a two-legged or multi-legged movingbody, or may be an air vehicle apparatus. The moving body 50 may not betrackless, and may be a line-tracing moving body which moves along adesignated track. Unless otherwise stated in the embodiment, the movingbody 50 is an AGV.

The moving body control unit 51 performs control of the drive mechanismfor moving the moving body 50 to a designated target position. Themoving body control unit 51 recognizes its position by the positionestimated at the position estimation device 1, and may control the drivemechanism by determining the necessary direction and distance for movingtowards the designated target position. The target position of themoving body 50 may be acquired from any one of: the host system of themoving body 50, e.g. the position estimation device 1, invoking a priorsetting, or a setting directly performed by a person. The targetposition is not only a spatial position of the moving body 50, and mayinclude a posture of the moving body 50 at the target position. Unlessotherwise stated in the embodiment, the position and the posture of thesensor mounted to the moving body 50 are the position and the posture ofthe moving body. When the target position is designated, specificposition coordinates may be specified and, through use of the mapinformation etc. of the operation, areas such as “position A” or“operation area B” may also be designated. Further, the instruction ofmovement is not limited to the instruction for designating the absoluteposition, and may be an instruction for designating a relative positionfrom the current position. For example, the instruction of movement maybe an instruction such as circling 30 degrees clockwise at a position 1m ahead. When moving a predetermined location, a movement path may alsobecome important. Even if the moving body tries to simply move linearlyto a predetermined position, there are cases where the movement becomesdifficult due to obstacles etc. The movement path may be decided by themoving body control unit 51 or provided by a superior system. Thus, themovement path may be provided by invoking a preset matter or by a directsetting made by a person.

The internal sensor of the sensor information collection unit 52 forexample, includes, a rotary encoder, an acceleration sensor, and anangular velocity sensor such as a gyro sensor. The movement amount andthe posture of the moving body 50 may be measured by the internalsensor. From the movement amount and the posture of the moving body 50,an estimated position of the moving body 50 can be acquired.

The external sensor of the sensor information collection unit 52collects the images outside of the moving body 50. It is preferable ifthe external sensor collects, in addition to the images outside themoving body 50, sensor information for measuring or calculating aperipheral distance of the moving body 50. It is further preferable ifthe external sensor collects sensor information for measuring orcalculating the peripheral distance of the moving body 50 at thesurface. For example, Depth Camera and LIDAR (Light Detecting andRaging) etc. can collect planar distance images. The distance image isan image generated by converting the value of distance to a value ofbrightness. The Depth Camera may be any method such as ToF (Time ofFlight) method, pattern irradiation method etc. Further, even a linemeasurement laser range finder, may collect planar distance images byaltering the mechanical measurement direction or a measurement directionfor multiple units. In addition, it is possible to utilize a stereocamera or single lens camera etc. as a method of indirect measurement.In the case of a stereo camera, images may be converted to distanceinformation by a stereo matching method. Even in the case of a singlelens camera, changes to the moving body position and posture can realizecapture akin to that of a stereo camera, rendering it possible tocalculate the distance information. The external sensor may be somethingother than the above. Unless otherwise stated in the embodiment, theexternal sensor is a stereo camera.

FIG. 3 shows an arrangement example of a stereo camera as one example ofan external sensor. In FIG. 3 , the stereo camera 521 is arranged on theupper central portion of the moving body 50. The stereo camera 521 isarranged so that its optic central 522 is at a position of height h froma ground 62. In addition, an optical axis of the stereo camera 521 isslanted for angle φ to the ground 62. Further, the angle of view of thestereo camera 521 is θx. A capture range 70 of the stereo camera 521 isa capture range 71 including a ceiling 60, which is mainly the positionof height H from the ground 62. As shown in FIG. 3 , when the movingbody 50 comes close to a wall 61, the capture range 70 of the stereocamera 521 becomes capture ranges 71 and 72 including the ceiling 60 andthe wall 61. The capture range 70 of the stereo camera 521 may bechanged depending on the usage of the images collected at the stereocamera 521. For example, when the usage is estimating the position ofthe moving body 50, it is preferable for the subject of the imagecollected at the stereo camera 521 to vary depending only on theposition of the moving body 50, and not temporally. As shown in FIG. 3 ,with the capture ranges 71 and 72 set to include the ceiling 60 and thewall 61, capture range 70 of the stereo camera 521 may collect imagessuitable for estimating the position of the moving body 50 for a subjectwith low temporal variation. The arrangement position and theorientation of the stereo camera 521 is not limited to those shown inFIG. 3 .

Next, the sensor information processing unit 10 will be furtherexplained. The evaluation information calculating unit 11 calculatesevaluation information associated with movement of the moving body 50from internal sensor information acquired at the sensor informationacquiring unit 21.

The information associated with the movement of the moving body 50 is,for example, the velocity of the moving body 50. In this case, theevaluation information calculating unit 11 calculates the amount ofmovement of the moving body 50 from a measurement value of the rotaryencoder as an internal sensor, attached to for example, wheels of themoving body 50. In addition, the evaluation information calculating unit11 calculates the velocity of the moving body 50 by differentiating thecalculated moving amount by time. The evaluation information calculatingunit 11 may calculate the velocity of the moving body 50 by convertingthe rotation speed of the drive mechanism, such as wheels etc., to avelocity of the moving body 50, or by calculating the velocity of themoving body 50 by converting the angular velocity of the drivemechanism, such as wheels etc., to the velocity of the moving body 50.Further, if an acceleration sensor is attached as an internal sensor tothe moving body 50, the evaluation information calculating unit 11 maycalculate the velocity of the moving body 50 by integrating theacceleration of the main body of the moving body 50. Further, theevaluation information calculating unit 11 may calculate the velocity ofthe moving body 50 from a drive current value etc. input from the movingbody control unit 51 to the drive mechanism of the wheels etc. In otherwords, the evaluation information calculating unit 11 may calculate thevelocity using sensor information of any type of internal sensor.

The information associated with the movement of the moving body 50 isnot limited to the velocity of the moving body 50. For example, theinformation associated with the movement of the moving body 50 may be arotational speed or angular velocity of the drive mechanism of wheelsetc. provided to the moving body 50, or acceleration of the moving body50.

The determination unit 12, by referring to the evaluation informationassociated with movement of the moving body 50 calculated at theevaluation information calculating unit 11, and evaluation informationassociated with an image of the target position registered in thedictionary storage unit 30, determines whether or not to adopt an imagecollected by the moving body 50 as an image for estimating a position ofthe moving body 50. More specifically, the determination unit 12, whenthe difference in the evaluation information is small, determines thatthe image collected at the moving body 50 is a suitable image forestimating a position of the moving body 50. At this point, thedetermination unit 12 determines to adopt the image collected at themoving body 50 as the image for estimating the position of the movingbody 50.

For example, the determination unit 12 determines if the image collectedat the moving body 50 is the suitable image for estimating the positionof the moving body 50, when a difference between the velocity of themoving body 50 calculated at the evaluation information calculating unit11 and the velocity associated with the image of the target positionregistered in the dictionary storage unit 30 is within a certain range.This range of velocity difference may be designated by a person, ordefined from the ratio to the velocity associated with the image of thetarget position registered to the dictionary storage unit 30. Forexample, the range of velocity difference may be set to 0±0.1 [m/s] fora state when the moving body 50 is stopped. In addition, for example, ifthe image of the target position registered in the dictionary storageunit 30 is captured when the velocity is 3 [m/s], the range of thevelocity difference is set to ±0.3 [m/s] which is its 10%.

The determination unit 12 may convert the velocity registered in thedictionary storage unit 30 to sensor information of the internal sensorsuch as the rotary encoder or the gyro sensor etc. and compares suchconverted sensor information with sensor information of the internalsensor. In this case, the determination unit 12 may determine that theimage collected by the moving body 50 is the suitable image forestimating the position of the moving body 50, when the differencebetween the sensor information acquired at the sensor informationacquiring unit 21 and sensor information converted from the velocityregistered in the dictionary storage unit 30 is within a predeterminedrange. For example, when the angular velocity converted from thevelocity registered to the dictionary storage unit 30 is 10 [rad/s], thedetermination unit 12 determines that the image should be used if thedifference of the angular velocity acquired by the sensor informationacquiring unit 21 being ±1 [rad/s].

In addition, when a plurality of images of the target position areregistered with the dictionary storage unit 30, the determination unit12 may determine the target position as either that designated by thesuperior system, or one set in advance.

Next, the dictionary storage unit 30 is explained. The dictionarystorage unit 30 stores a reference dictionary in which informationnecessary to the target position at the sensor information processingunit 10 and the position estimation unit 22 is registered. Theregistered information of the reference dictionary, for example, is thefollowing.

-   -   Images of target position and its surroundings    -   Feature points and feature amounts of images of target position        and its surroundings    -   Effect of correspondence between the images    -   Positions of target position and its surroundings    -   Evaluation information (for example, velocity) associated with        movement of moving body during capture of the image of the        target position.

There are two ways of registering with the reference dictionary when aplurality of target positions exists, 1) a method of collecting theregistration information for all target positions and registering to onedictionary, or 2) a method separating the dictionary for each targetposition. The method 1) will not require the selection of a dictionary;however, as it becomes necessary to match the feature points for allimages registered in the reference dictionary, the process of estimatingthe position will become time-consuming. Further, as it can be completedmerely by the process of registering the necessary target position withthe reference dictionary, the process time for estimating the positionin method 2) is shorter. On the other hand, in the method 2), it isnecessary to designate the reference dictionary to be used by a hostsystem or person etc. As can be seen from the above reference dictionaryregistration methods 1) and 2), both have their advantages anddisadvantages. Thus, it is preferable if the registration methods 1) and2) for the reference dictionary are used in different situations whennecessary.

FIG. 4 is a flowchart indicating a process when preparing a referencedictionary. The process of FIG. 4 will be performed before theestimation of the position.

In step S101, the control unit 23 of the position estimation device 1starts moving the moving body 50 to one designated position. Thedesignated position includes the target position and its surroundingcapture positions. The control unit 23 selects one of the above andinstructs the moving body 50 to move. With this instruction, the movingbody 50 starts autonomous movement. The movement of the moving body 50may be performed manually by the operation of a joystick etc.

In step S102, the control unit 23 determines, from sensor informationacquired at the sensor information acquiring unit 21, whether or not themoving body 50 has stopped. The control unit 23, for example, calculatesthe velocity of the moving body 50 from the sensor information acquiredat the sensor information acquiring unit 21, and determines whether themoving body 50 has stopped when the calculated velocity is equal to orlower than a threshold. For the velocity of the moving body 50, thevelocity calculated at the evaluation information calculating unit 11may be used. Here, the stoppage of the moving body 50 is not limited tothe arrival of the target position or the capture positions. Forexample, the moving body 50 may be configured stop at a corner etc. whenmoving towards the target position or the capture positions. In thedetermination of step S102, even for the above case, if the velocity ofthe moving body 50 is equal to or lower than the threshold, it isdetermined that the moving body 50 has stopped. In step S102, theprocess is on standby until it is determined that the moving body 50 hasstopped. In step S102, when it is determined that the moving body 50 hasstopped, the process proceeds to step S103.

In step S103, the control unit 23 instructs the moving body 50 tocapture and acquires an image from the moving body 50.

In step S104, the control unit 23 determines whether or not to adesignated number of images have been acquired. In an exemplaryinstance, the control unit 23 determines that the designated number ofimages have been acquired when the images of the target position and allcapture positions have been acquired. In step S104, if the designatednumber of images are not acquired, i.e., if there is still the captureposition in which the image has not been acquired, the process willreturn to step S101. In this case, the control unit 23 will designate anew capture position and start the movement of the moving body 50. Instep S104, if a designated number of images are acquired, the processproceeds to step S105.

In step S105, the position estimation unit 22 detects the feature pointsof the acquired images. The position estimation unit 22 may detect thefeature points using SIFT (Scale Invariant Feature Transform), AKAZE(Accelerated KAZE) etc.

In step S106, the position estimation unit 22 extracts feature amountsfrom the respective detected feature points. In the position estimationunit 22, the feature amounts may be extracted in accordance with themethod used in the feature point detection.

In step S107, the position estimation unit 22 performs feature pointmatching for the target position image and the image at each captureposition. More specifically, the position estimation unit 22 associatesthe feature points between the target position image and the image ateach capture position so that the difference in the feature amounts willbe the smallest. The position estimation unit 22 may perform featurepoint matching at the method of NN (Nearest Neighbor), k-NN, kd-Tree,humming distance etc.

In step S108, the position estimation unit 22 performs estimation ofcorrespondence between a target position image and an image of eachcapture position. For example, the position estimation unit 22 performsestimation of relative position and relative posture of the moving body50 at each capture position to the target position from a correspondenceof feature points between a target position image and each captureposition image. In addition, the position estimation unit 22 generatesthree-dimensional information for each image by a triangulationprinciple using the designated relative position and relative posture.

In step S109, the position estimation unit 22 registers to the referencedictionary: feature points and feature amounts for every image,correspondence between the images of each capture position and thetarget position, coordinates of the target position and the capturepositions, and evaluation information associated with the movement ofthe moving body 50 when collecting respective images, e.g., velocity ofthe moving body 50.

FIG. 5 is a diagram showing an example of a capture positions of animage to be registered to the reference dictionary in accordance withthe process of FIG. 4 . In FIG. 5 , a position directly in front of acage truck position RBP is set to be a target position P0. In addition,positions P1, P2, P3, P4 and P5 surrounding the target position P0 areset to capture positions. At the capture positions P1, P2, P3, P4 andP5, capture is taken with the stereo camera directed to the targetposition P0. The target position P0 is a known position provided by, forexample, a superior system etc. On the other hand, the capture positionsP1 to P5 are positions for performing measurements each time capture istaken at the respective positions.

Here, the number of images registered to the reference dictionary, i.e.,the number of capture positions, is not limited to a specific value.Furthermore, in principle, no problem is posed by any type of positionalrelationship between the target position and each capture position. In apractical sense, it is desirable that the capture positions arepositioned within a vicinity range NB of the target position.

Next, the position estimation unit 22 is explained. The positionestimation unit 22 estimates a position of the moving body 50 bycomparing the image among those acquired at the sensor informationacquiring unit 21 that is determined, by the determination unit 12 ofthe sensor information processing unit 10, as being adoptable forestimation of a position of the moving body 50, with an image registeredto the reference dictionary of the dictionary storage unit 30. FIG. 6 isa flowchart indicating a process during position estimation.

In step S201, the sensor information acquiring unit 21 acquires sensorinformation, e.g., a measurement value of a rotary encoder, of aninternal sensor from the sensor information collection unit 52 of themoving body 50, and sensor information, e.g., an image obtained by astereo camera, of an external sensor. The sensor information acquiringunit 21 acquires sensor information, from the sensor informationcollection unit 52, of the moving body 50 at, for example, everyconstant interval.

In step S202, the evaluation information calculating unit 11 calculatesevaluation information from sensor information of the internal sensoracquired at the sensor information acquiring unit 21. For example, theevaluation information calculating unit 11 calculates the velocity ofthe moving body 50 from the measurement value of the rotary encoder.

In step S203, the determination unit 12 determines whether or not theimage acquired from the sensor information acquiring unit 21 is suitablefor estimating the position of the moving body 50. For example, thedetermination unit 12 determines if the image collected at the movingbody 50 is the suitable image for estimating the position of the movingbody 50, when a difference between the velocity of the moving body 50calculated at the evaluation information calculating unit 11 and thevelocity associated with the image of the target position registered inthe dictionary storage unit 30 is within a certain range. In step S203,the process of FIG. 6 is terminated when the image acquired from thesensor information acquiring unit 21 is determined as unsuitable for theestimation of the position of the moving body 50. The process of FIG. 6is performed once again at a timing of acquiring the next sensorinformation by the sensor information acquiring unit 21. In step S203,when the image acquired from the sensor information acquiring unit 21 isdetermined as suitable for estimating the position of the moving body50, the process proceeds to step S204.

In step S204, the position estimation unit 22 acquires, from the sensorinformation acquiring unit 21, the image determined for adoption as theposition estimate deemed suitable for such a purpose in step S203.Further, the position estimation unit 22 detects the feature points fromthe acquired image. The position estimation unit 22 may detect thefeature points using SIFT (Scale Invariant Feature Transform), AKAZE(Accelerated KAZE) etc.

In step S205, the position estimation unit 22 extracts feature amountsfrom the respective detected feature points. In the position estimationunit 22, the feature amounts may be extracted in accordance with themethod used in the feature point detection.

In step S206, the position estimation unit 22 performs feature pointmatching for the target position image and images of surrounding capturepositions registered to the reference dictionary stored in thedictionary storage unit 30, as well as for the image acquired at thesensor information acquiring unit 21. More specifically, the positionestimation unit 22 performs the feature point matching so that thedifference in feature amount will be at the minimum. When a plurality oftarget positions are registered to the reference dictionary, it isnecessary to select the target position. The target position may beselected by either 1) a method of acquiring a target position from ahost system mounted on the moving body, or 2) a method of estimatingpositions of the moving body 50 to all target positions registered tothe reference dictionary and selecting the best estimation result.

In step S207, the position estimation unit 22 calculates the relativeposition using PnP (Perspective n-Point) from the relationship betweenthree-dimensional point groups and its corresponding two-dimensionalpoints of feature points registered to the reference dictionary.Further, the position estimation unit 22 estimates a position of themoving body 50 from the target position. Afterwards, the process of FIG.6 is terminated. The process of FIG. 6 is performed once again at atiming when the next sensor information is acquired by the sensorinformation acquiring unit 21.

In the first embodiment as explained above, when the difference betweenevaluation information based on sensor information collected at theinternal sensor included by the moving body 50 and evaluationinformation based on sensor information at the time of collecting theimage registered to the reference dictionary is small, the imagecollected by the external sensor included to the moving body 50 isadopted as the image for estimating the position of the moving body 50.

The precision of the position of the moving body 50 estimated at theposition estimation unit 22 changes greatly depending on whether or notthe capture conditions are considered.

FIG. 7A is a diagram showing an example when feature point matching isperformed at images with different capture conditions. The image RI ofFIG. 7A is an image registered to the reference dictionary, and an imageSI is an image collected for position estimation. There is a possibilitythat the capture conditions differ from the time of collecting for theposition estimation and the time of registration to the referencedictionary. For example, the image SI in FIG. 7A is blurred in a waythat does not occur in image RI due to the shaking of the moving body 50at the time the capture was taken. In this case, as a result of matchingbetween image RI and image SI, there is a possibility that matching mayoccur at a position different from the position of the original featurepoints to be matched. In FIG. 7A, a feature point RF1 and a featurepoint SF11, a feature point RF2 and a feature point SF21, a featurepoint RF3 and a feature point SF31, and a feature point RF4 and afeature point SF41 should be matched; however, the figure indicates thefeature point RF1 and the feature point SF12, the feature point RF2 andthe feature point SF22, the feature point RF3 and the feature pointSF32, and the feature point RF4 and the feature point SF42 as beingmatched. The false matching results of the feature points lowers theestimation accuracy of the position.

FIG. 7B is a diagram showing an example when feature point matching isperformed at images with similar capture conditions. The image RI ofFIG. 7B is an image registered to the reference dictionary, and an imageSI is an image collected for position estimation. Since the capturecondition is the same at both the time during the collection forposition estimation and the time when images are registered to thereference dictionary, matching is performed at the position of thefeature points that should be matched such as the feature point RF1 andthe feature point SF1, the feature point RF2 and the feature point SF2,the feature point RF3 and the feature point SF3, the feature point RF4and the feature point SF4. In this way, in the first embodiment,preventative measures against causes of decreased position estimationaccuracy, such as stereo camera shaking etc., may be taken, and themoving body 50 position estimation process may be performed with featurepoint matching from high quality images. Therefore, the positionestimation accuracy of the moving body 50 improves.

Further, in the embodiment, the image registered to the referencedictionary is captured when the velocity of the moving body 50 is equalto or lower than the threshold. Thus, it is considered that the blurringof images registered to the reference dictionary is kept to a minimum.Furthermore, if the moving body 50 is assumed to move autonomously, lossof speed often means it is at a position to be focused on, such as, forexample, near the target position, a corner, or a dead end etc. Ameaningful position estimation of the moving body 50 is performed byperforming capture at such positions.

In the first embodiment, the moving body 50 and the position estimationdevice 1 are described as separate systems. However, the moving body 50and the position estimation device 1 may be integral. Further, a part ofthe position estimation device 1 may be separate from the positionestimation device 1. For example, the dictionary storage unit 30 may beprovided with an external server outside the position estimation device1.

Second Embodiment

FIG. 8 is a diagram showing a configuration of one example of a systemaccording to a second embodiment. Similar to the first embodiment, thesystem includes a position estimation device 1 and a moving body 50. Theposition estimation device 1 is configured to be communicable with themoving body 50. In the following explanation, for the parts similar tothose of the first embodiment, the same reference numbers are added anddetailed descriptions thereof are omitted. The second embodiment differsfrom the first embodiment only in that the position estimation device 1includes a processing unit 24. The processing unit 24 is achieved by,for example, the CPU 101.

The processing unit 24 compares an image of the target positionregistered to the reference dictionary of the dictionary storage unit 30with an image of the target position collected at a different time inthe moving body 50, and detects a defective part in the target position.The defective part is a part that does not originally exist in thetarget position such as holes, cracks, and scratches etc. on the walls.FIG. 9 is a diagram showing a detection example of the defective part bythe processing unit 24.

As previously explained, the position estimation unit 22 of theembodiment estimates the position of the moving body 50 using an imageadopted by the determination unit 12. Therefore, when the position ofthe moving body 50 estimated by the position estimation unit 22 is thetarget position, the capture conditions of an image SI, acquired at thesensor information acquiring unit 21, are similar to the captureconditions of an image RI of the target position registered to thereference dictionary, meaning that image RI and the image SI matchesbasically match. On the other hand, when the defective part is in thetarget position, as shown in FIG. 9 , it will be indicated as thedifference between the image RI and the image SI.

As a detection process of the defective part by the processing unit 24,various processes may be applied, such as: corresponding two imagesusing feature point matching, and having the part with the greatestdifference indicated as the defective part; detecting a part with a lowdegree of similarity found by template matching as the defective part;detecting a part with the biggest difference in brightness by abackground subtraction method.

As explained above, according to the second embodiment, the detection ofa defective part is performed using high quality images. Thus, variousapplications of the moving body 50 using the maintenance and inspectioncan be anticipated.

Similar to the first embodiment, the moving body 50 and the positionestimation device 1 may be integral in the second embodiment. Further, apart of the position estimation device 1 may be separate from theposition estimation device 1. For example, the dictionary storage unit30 may be provided with an external server outside the positionestimation device 1.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

The invention claimed is:
 1. A position estimation device comprising:processing circuitry configured to: acquire an external image of amoving body from an external sensor installed in the moving body;acquire first evaluation information associated with a movement of themoving body collected by an internal sensor installed in the moving bodywhen collecting the external image at the external sensor; acquire forposition estimation at least one reference image associated with acollection position, the reference image having previously beencollected under a same condition of the external image and in advance ofacquisition of the external image, including, determine to adopt theexternal image as a first image for estimating a position of the movingbody when a difference between the first evaluation information andsecond evaluation information of the reference image is within a certainrange, the reference image being among a plurality of potentialreference images and respective evaluation information registered in adictionary storage unit; and estimate a position of the moving body bycomparing the first image and the reference image.
 2. The positionestimation device according to claim 1, wherein the processing circuitryis further configured to detect a part different from the referenceimage in the first image as a defective part by comparing the firstimage and the reference image collected at a same collection position.3. The position estimation device according to claim 1, wherein thefirst evaluation information and the reference evaluation informationinclude at least either of a velocity of the moving body, and an angularvelocity of the moving body.
 4. The position estimation device accordingto claim 1, wherein the reference image is an image captured when anevaluation information associated with the movement of the moving bodyis equal to or lower than a threshold.
 5. A system comprising: a movingbody comprising: an external sensor configured to collect an externalimage of the moving body; and an internal sensor configured to collectevaluation information associated with a movement of the moving body;and a position estimation device comprising processing circuitryconfigured to: acquire for position estimation at least one referenceimage associated with a collection position, the reference image havingpreviously been collected under a same condition of the external imageand in advance of acquisition of the external image, including,determine to adopt the external image as a first image for estimating aposition of the moving body when a difference between first evaluationinformation associated with a movement of the moving body collected bythe internal sensor when collecting the external image at the externalsensor and second evaluation information of the reference image iswithin a certain range, the reference image being among a plurality ofpotential reference images and respective evaluation informationregistered in a dictionary storage unit; and estimate a position of themoving body by comparing the first image and the reference image.
 6. Aposition estimation method comprising: acquiring an external image of amoving body from an external sensor installed in the moving body;acquiring first evaluation information associated with a movement of themoving body collected by an internal sensor installed in the moving bodywhen collecting the external image at the external sensor; acquiring forposition estimation at least one reference image associated with acollection position, the reference image having previously beencollected under a same condition of the external image and in advance ofacquisition of the external image; determining to adopt the externalimage as a first image for estimating a position of the moving body whena difference between the first evaluation information associated withthe external image is within a predetermined range of second evaluationinformation associated with the reference image, the reference imagebeing among a plurality of potential reference images and respectiveevaluation information being registered in a dictionary storage unit;and estimating a position of the moving body by comparing the firstimage and the reference image.